# English Pretraining

T5-Efficient-SMALL is a variant of Google's original T5, adopting a deep narrow architecture that outperforms other architectures in downstream tasks with similar parameter counts.

T5-Efficient-LARGE is a variant based on Google's T5, optimized for downstream task performance with a deep narrow architecture, featuring 737.7 million parameters.

T5-Efficient-TINY-FF12000 is a variant of Google's original T5, adopting a deep narrow architecture that demonstrates superior downstream task performance among models with similar parameter counts.

T5-Efficient-BASE is a variant of Google's T5 architecture, featuring a deep-narrow design optimized for downstream task performance, with 222.9 million parameters.

T5 Efficient Small-NL22 is a deep narrow variant of Google's T5 model, focusing on improving downstream task performance by increasing model depth.

T5-Efficient-TINY is a deep-narrow variant of Google's T5 model, focusing on improving downstream task performance by increasing model depth rather than width.

T5-Efficient-BASE-FF6000 is a variant of Google's original T5, adopting a deep narrow architecture that demonstrates superior downstream task performance among models with similar parameter counts.

T5 Efficient Large-DM2000 is a variant of Google's T5 model, adopting a deep narrow architecture that prioritizes increasing model depth to enhance downstream task performance.